Method for Regenerating the Background of Digital Images of a Video Stream

ABSTRACT

The invention relates to a method for regenerating the background of digital images of a video stream comprising steps consisting in: -setting an initial background image, -cutting the unit images of the video stream into blocks b (i, j, t) and of the background image into corresponding blocks Bo (i, j, t). The method is essentially characterized in that it further includes steps consisting in: -selecting one block Bo of the background image and/or b of the frame image, and •calculating the space correlation thereof, with: •at least one block Bo of the background image at a time (t), and/or at another time (t−a), and/or .at least one block b of the frame image at a time (t), and/or at another time (t−a), and/or -updating the background image according to the calculation of the space correlation.

The present invention relates to the field of the automatic processingof digital images, more particularly for retrieving a stationarybackground or a static image, from a plurality of frames.

Frames are captured by a video camera or a picture camera, possiblyintegrated in a communicating object (telephone, PDA, etc.), hereinafterknown by the generic word “camera”.

It is assumed that the camera is stationary, that the scene shot has atleast one stationary background, for example corresponding to astationary subject (an object, a building, a character, etc . . . ), andpreferably, that light is substantially identical when the pictures aretaken. Objects which are moving with respect to such a stationarybackground are designated as the foreground.

Methods for regenerating the background of digital images of a videostream produced by a fixed camera and including a plurality oftime-sequential unit images are known and enable, for example, tofollow-up moving objects for safety applications, or the digitalediting, either for down-breaking the movement, or for the eliminationof undesirable moving subjects in the foreground.

However, such methods generally require the capture of aninterference-free background image in the foreground, refer for exampleto the document US 2003/0194131, which is not always possible.

In addition, most of such processes are exclusively based on timecorrelations, which means that they have processing defects, moreparticularly when the scene includes a partially stationary object inthe foreground. As a matter of fact, the stationary part of the objectis then considered as being integral with the background, which pollutesthe regenerated background image.

The present invention aims at improving the existing techniques.

For this purpose, the invention relates, as per the first of itsobjects, to a method for regenerating the background of digital imagesof a video stream produced by a fixed camera and including a pluralityof time-sequential frames, with each one including a respective unitimage,

the method comprising steps consisting in:

-   -   setting an initial background image in a memory,    -   cutting the unit images of the video stream into blocks (i,        j, t) of at least one pixel, and    -   cutting the background image into corresponding blocks Bo(i,        j, t) having the same size and the same coordinates (i, j) as        those of the unit images.

According to the invention, the method is essentially characterized inthat it further includes steps consisting in:

-   -   selecting at least one block b(i, j, t), b(i, j, t−a), Bo(i, j,        t−a), and Bo(i, j, t), and    -   calculating the space correlation thereof, with the space        correlation consisting in computing:        -   the space correlation (Corr_spat_mes_b, Corr_spat_mes_b_Bo)            between a given block b(i, j, t) of the frame image, at a            time (t), and        -   at least one block Bo(i±x*δi, j±y*δj, t) of the background            image close to said block Bo(i, j, t), at a time (t), and/or        -   at least one block Bo(i±x*δi, j±y*δj, t−a) of the background            image close to said block Bo(i, j, t−a), at another time            (t−a), and/or        -   at least one block b(i±x*δi, j±y*δj, t) of the frame image            close to said block b(i, j, t), at a time (t), and/or the            space correlation (Corr_spat_mes_Bo, Corr_spat_mes_Bo_b)            between a given block Bo(i, j, t) of the background image,            at a time (t), and        -   at least one block Bo(i±x*δi, j±y*δj, t) of the background            image close to said block Bo(i, j, t), at a time (t), and/or        -   at least one block Bo(i±x*δi, j±y*δj, t−a) of the background            image close to said block Bo(i, j, t−a), at another time            (t−a), and/or        -   at least one block b(i±x*δi, j±y*δj, t) of the frame image            close to said block b(i, j, t), at a time (t), and    -   updating the background image according to the calculation of        the space correlation.

Thus, the proposed solution searches background block considering amixed temporal and spatial pixel value correlation criterion: abackground pixel block is updated if its pixel values are very similar(temporally correlated) to collocated previous pixel blocks in a pastscanned sequence frames but also if they are highly similar (spatiallycorrelated) to neighbor pixels blocks in the background map.

Preferably, the updating of the background image includes a stepconsisting in substituting the block b(i, j, t), b(i, j, t−a), Bo(i, j,t−a) or Bo(i, j, t) for the block Bo(i, j, t) by selecting among thesethe one having the minimum space correlation.

Preferably, the space correlation is calculated by measuring the sum ofthe absolute differences (SAD) between the blocks considered, thus theinvention considers a spatial correlation working in the pixel domainbased on SAD.

In one embodiment, the space correlation between a given block ((b(i, j,t), Bo(i, j, t)) and a set of blocks (b(i±x*δi, j±y*δj, t), Bo(i±x*δi,j±y*δj, t)), with {x, y}ε{1;2; . . . ;Δ}² is set along specificdirections,

with the minimum value of the sums of the absolute differences (SAD)along each one of these specific directions being retained as a measureof the space correlation.

In one embodiment, the method further comprises steps consisting in:

-   -   calculating the time correlation (Corr_temp_mes_b) between a        given block b(i, j, t) of an image at a time (t) and the same        block b(i, j, t−a) of one image or of several images at a        preceding time (t−a), and/or    -   calculating the time correlation (Corr_temp_mes_Bo) between a        given block Bo(i, j, t) of an image at a time (t) and the same        block Bo(i, j, t−a) of one image or of several images at a        preceding time (t−a), and    -   updating the background image also according to the time        correlation.

In one embodiment, the space correlation coupled to a measure of thetime correlation is measured (sum1) on at least two frames, where theweight of the space correlation is weighted by a parameter λ dependingon time λ(t), so that

Sum1=Corr_temp_mes_(—) b(i, j, t)+λ(t)·Corr_spat_mes_(—) b_Bo(i:i±x*δi,j:j±y*δj, t:t),

the space correlation coupled to a measure of the time correlation ismeasured (sum2) on at least two frames, where the weight of the spacecorrelation is weighted by a parameter λ depending on time λ(t), so that

Sum2=Corr_temp_mes_Bo(i, j, t)+λ(t)·Corr_spat_mes_Bo(i:i±x*δi, j:j±y*δj,t−a:t),

andthe value of the block Bo(i, j, t) is updated by selecting the minimumvalue of both values (Sum1, Sum2).

In one embodiment, each frame of blocks b(i, j) or background image ofblocks (Bo(i, j) is scanned at least in a first and in a seconddirection, so that a first series of correlation computations isperformed in the first direction; and a second series of correlationcomputations is performed in the second direction.

For example, the size of the blocks b(i, j) or Bo(i, j) in the firstdirection is different from that of the blocks b(i, j) or Bo(i, j) inthe second direction.

In one embodiment, the method according to the invention previouslyfurther comprises a step consisting in computing the sum of measures ofstationarity for the set of the blocks b(i, j, t) of a frame at a time(t) and

-   -   comparing the sum of the measures of stationarity with a first        threshold (T1), and    -   if the sum is under the first threshold (T1), then the frame is        not taken into account for the space correlation computations;        and/or    -   comparing the sum of the measures of stationarity with a second        threshold (T2), and    -   if the sum is above the second threshold (T2), then the frame is        not taken into account for the space correlation computations.

In one embodiment, the method according to the invention furthercomprises steps consisting in:

-   -   calculating the speed of convergence of the background image,    -   comparing the calculated speed with a threshold, and    -   stopping the scanning of the blocks Bo(i, j) of the background        image if the speed is under the threshold.

According to another one of the objects, the invention also relates to acomputer programme, including programme code instructions for theexecution of the steps of the method defined hereabove, when saidprogramme is executed by a computer.

Thanks to the invention, the background image is updated on the fly,which makes it available and thus displayable at any time, whichimproves the latency as compared to the techniques of the prior art,which require all the frames. Similarly, thanks to the progressiveconstruction of the background image thereof, the invention makes itpossible to optimise the memory resources; whereas the prior artrequires to store the whole video.

Another advantage relies on that the proposed solution is a single-mapbackground reconstruction: it only stores in memory one backgroundimage, whose pixel blocks are updated along the sequence frames scan.

It also starts the background reconstruction process with a backgroundmap possibly containing error background elements (which will be updatedduring the background block research), and being thus entirely filled.

Other characteristics and advantages of the present invention willappear more clearly upon reading the following description which isgiven as an illustrative and not a limitative example, and referring tothe appended drawings, wherein:

FIG. 1 illustrates the cutting into blocks of each video frame and thecutting into corresponding blocks of the background image,

FIG. 2 illustrates the time correlation between identical blocks atdifferent times,

FIG. 3 illustrates one embodiment of the method according to theinvention,

FIG. 4 illustrates one embodiment of the method according to theinvention,

FIG. 4′ illustrates one embodiment of the method according to theinvention,

FIG. 4″ illustrates one embodiment of the method according to theinvention,

FIG. 4′″ illustrates one embodiment of the method according to theinvention,

FIG. 5 illustrates one embodiment of the method according to theinvention,

FIG. 5′ illustrates one embodiment of the method according to theinvention.

According to the present invention, it is assumed that the camera isfixed for the duration of the exposition.

A first step consists in capturing a plurality of time-sequential unitimages of a scene.

Another step consists in setting an initial background image.

Each unit image of the video stream corresponding to a frame emitted ata time t, is cut into regular blocks b(i, j, t) of at least one pixel(FIG. 1). Similarly, the background image is cut into correspondingregular blocks Bo(i, j, t), i.e. one block Bo (i, j) of the backgroundimage has the same coordinates as a block b (i, j) of the frame image.

In one embodiment, each image is divided into N blocks. For example,N=40.

The size of a block preferably depends on the image resolution and/orthe size of the moving objects in the foreground.

From the blocks, refer to FIG. 2, the time correlationCorr_temp_mes_b(i, j, t:t−a) between a given block b(i, j, t) of animage at a time (t) and the same block b(i, j, t−a) of one image orseveral images at another time (t−a), and/or the time correlationCorr_temp_mes_Bo(i, j, t:t−a) between a given block Bo(i, j, t) of abackground image at a time (t) and the same block Bo(i, j, t−a) of oneimage or several background images at another time (t−a).

According to the invention, the time (t−a) can be before (a>0) or after(a<0) time t. Two consecutive or not consecutive frames can be selected,so that, if two consecutive frames are separated by δt, the inter-framegap, then |a| is a multiple of δt.

Preferably, the time correlation between the block b(i, j, t) and thesame block b(i, j, t−a) is determined between two consecutive frames,which shortens the computation times. Then, a=δt.

This measure of time correlation is also called the measure ofstationarity, of block b(i, j, t) or block Bo(i, j, t). The results ofthis measure are saved in a memory. This measure Corr_temp_mes_b(i, j,t:t−a), Corr_temp_mes_Bo(i, j, t:t−a) is made and used as describedhereinunder.

Preferably, the method according to the invention further comprises astep consisting in calculating the sum of the measures of stationarityfor all the blocks b(i, j, t) of one frame at a time t. This sumCorr_temp_mes_b(ΣI, Σj, t:t−a) for a given frame is then compared to afirst threshold T1. If the sum is under the threshold T1, then the frameis not taken into account. As a matter of fact, too high a level ofstationarity corresponding to too low measures of stationarity mayinterfere with the computations according to the invention.

Similarly, the method according to the invention may further comprise astep consisting in comparing the sum Corr_temp_mes_b(ΣI, Σj, t:t−a) fora given frame at a second threshold T2. If the sum is above the secondthreshold T2, then the frame is not taken into account, either. As amatter of fact, too low a level of stationarity corresponding to toohigh measures of stationarity may mean that the number and/or themobility of the objects in the foreground is too high and interfereswith the computations.

Preferably, the respective value of the thresholds T1 and T2, depends onthe size of the image, i.e. the number of pixels in the image, or evenon the size of the blocks.

The initial background image of the scene, also called the static image,is saved in a memory. The initial background image may be one of theunit images, for example the first one or the one showing the leastmotions, or else a background image built from exclusively timecorrelation measures, as described hereabove.

At any time t, we simultaneously have a frame cut into blocks b(i, j, t)and a corresponding background image cut into blocks Bo(i, j, t), thevalues of which are a minimum equal to those of the blocks Bo(i, j,t−δt) of the background image at the previous time t−δt. The backgroundimage is thus updated on the fly and available at any time.

The background image is then possibly updated at a time t as a functionof the respective values of the blocks Bo(i, j) and b(i, j) at a time tor at another time t−a, as described hereinafter.

According to the invention, the method further comprises a stepconsisting in determining the space correlation between blocks close andpreferably adjacent to one another. For example, when the shape of theblocks is square or rectangular, two adjacent blocks have a side and/oran angle in common (i.e. a diagonal in common). And more generally, twoblocks including K×K pixels are close if the baric centres of saidblocks are spaced at most by K pixels, more particularly if both blockshave an adjacent side, or K times the square root of two, if the blockshave an angle in common.

Then, as described hereinafter, the space correlation can be determined:

-   -   between a given block b(i, j, t) of a frame at a time (t) and at        least another block b(i+δi, j+δj, t) of the same frame; and/or    -   between a given block b(i, j, t) of a frame at a time (t) and at        least a block Bo(i+δi, j+δj, t)—preferably other than block        Bo(i, j, t)—of the corresponding background image; and/or    -   between a given block Bo(i, j, t) of the background image at a        time (t) and at least another block Bo(i+δi, j+δj, t) of the        background image.

For this purpose, in one embodiment, the sum of the space correlationsCorr_spat_mes_b(i:i±x*δi, j:j±y*δj, t:t) between a given block b(i, j,t) of one image at a time (t) and at least one preferably adjacentblock, b(i±x*δi, j±y*δj, t), of the same frame is calculated, refer toFIG. 3.

In order to simply illustrate FIG. 3, only the space correlationCorr_spat_mes_b(i:i, j:j+2δj, t:t) between the block b(i, j, t) and theblock b(i, j+2δj, t) and the space correlation Corr_spat_mes_b(i:i,j:j−δj, t:t) between the block b(i, j, t) and the block b(i−δi, j, t)have been represented.

Similarly, as also illustrated in FIG. 3, according to the invention,the sum of the space correlations Corr_spat_mes_Bo(i:i±x*δi, j:j±y*δj,t−a:t−a) between the block Bo(i, j, t−a) of the background image at atime (t) corresponding to said given block b(i, j, t) of the image at atime t, and at least a close and preferably adjacent block Bo(i±x*δi,j±y*δj, t−a) of the background image at the same time (t), possibly witha=0.

Preferably, the respective value of x and y is low, so as to keep thenotion of proximity. For example, x=y=2.

In order to give a simple illustration of FIG. 3, the space correlationCorr_spat_mes_Bo(i:i, j:j+2δj, t−a:t−a) between the block Bo(i, j, t−a)and the block Bo(i, j+2δj, t−a) and the space correlationCorr_spat_mes_b(i:i, j:j−δj, t−a:t−a) between the block Bo(i, j, t) andthe block Bo(i−δi, j, t−a) only have been represented, possibly witha=0.

Then, the value of the block Bo(i, j, t) is updated by selecting amongthe blocks b(i, j, t) and Bo(i, j, t) the one the coordinates of whichhave the minimum value of the two following sums:

Bo(i, j, t)

min [Corr_spat_mes_b(i:i±x*δi, j:j±y*δj, t:t);Corr_spat_mes_Bo(i:i±x*δi, j:j±y*δj, t−a:t−a)].

In another embodiment, the space correlation between a block of theframe and at least a corresponding block adjacent to the backgroundimage can be calculated.

For this purpose, the sum of the space correlationsCorr_spat_mes_b_Bo(i:i±x*δi, j:j±y*δj, t:t) between a given block b(i,j, t) of an image at a time (t) and at least a block Bo(i±x*δi, j±y*δj,t) of the background image, preferably another one than the block Bo(i,j, t) close to, and preferably adjacent to said block Bo(i, j, t) at thesame time (t) is calculated, refer to FIG. 4.

As also illustrated in FIG. 4, the sum of the space correlationsCorr_spat_mes_Bo(i:i+x*δi, j:j±y*δj, t:t) between the block Bo(i, j, t)of the background image at a time (t) corresponding to said given blockb(i, j, t) of the image at a time t, and at least a block, close to andpreferably adjacent to Bo(i±x*δi, j±y*δj, t) of the background image atthe same time (t) is also measured.

In order to simply illustrate FIG. 4, the space correlationCorr_spat_mes_Bo(i:i, j:j+2δj, t:t) between the block Bo(i, j, t) andthe block Bo(i, j+2δj, t) and the space correlationCorr_spat_mes_Bo(i:i, j:j−δj, t:t) between the block Bo(i, j, t) and theblock Bo(i−δi, j, t) only have been represented.

The value of the block Bo(i, j, t) is then updated by selecting amongthe blocks b(i, j, t) and Bo(i, j, t), the one, the coordinates of whichhave the minimum value of these two sums:

Bo(i, j, t)<=>min[Corr_spat_mes_(—) b_Bo(i:i±x*δi, j:j±y*δj, t:t);Corr_spat_mes_Bo(i:i±x*δi, j:j±y*δj, t:t)].

Similarly, in another embodiment illustrated in FIG. 4′, the sum of thespace correlations Corr_spat_mes_b_Bo (i:i±x*δi, j:j±y*δj, t:t) betweena given block b(i, j, t) of an image at a time (t) and at least oneblock Bo(i±x*δi, j±y*δj, t) of the background image, preferably otherthan the block Bo(i, j, t), close to, and preferably adjacent to saidblock Bo(i, j, t) at the same time t is calculated.

As also illustrated in FIG. 4′, the sum of the space correlationsCorr_spat_mes_b(i:i+x*δi, j:j±y*δj, t:t) between the block b(i, j, t) ofthe frame image at a time (t), and at least one block, close to, andpreferably adjacent to said block, b(i±x*δi, j±y*δj, t) of the frameimage at the same time (t) is also measured.

In order to simply illustrate FIG. 4′, only the space correlationCorr_spat_mes_b(i:i, j:j+2δj, t:t) between the block b(i, j, t) and theblock b(i, j+2δj, t), and the space correlation Corr_spat_mes_b(i:i,j:j−δj, t:t) between the block b(i, j, t) and the b(i−δi, j, t) havebeen represented.

The value of the block Bo(i, j, t) is then updated by selecting amongthe blocks b(i, j, t) and Bo(i, j, t), i.e. the one the coordinates ofwhich have the minimum value of these two sums:

Bo(i,j, t)<=>min[Corr_spat_mes_(—) b_Bo(i:i±x*δi, j:j±y*δj, t:t);Corr_spat_mes_(—) b(i:i±x*δi, j:j±y*δj, t:t)].

Similarly in another embodiment illustrated in FIG. 4″, the sum of thespace correlations Corr_spat_mes_Bo_b (i:i±x*δi, j:j±y*δj, t:t) betweena given block Bo(i, j, t) of the background image at a time (t) and atleast one block b(i±x*δi, j±y*δj, t) of the image of the video frame atthe same time t, preferably another one than the block b(i, j, t), closeto, and preferably adjacent to said block b(i, j, t) is calculated.

As also illustrated in FIG. 4″, the sum of the space correlationsCorr_spat_mes_b(i:i±x*δi, j:j±y*δj, t:t) between the block b(i, j, t) ofthe frame image at a time (t), and at least one block, close to, andpreferably adjacent to said block b(i±x*δi, j±y*δj, t) of the frameimage at the same time (t) is also measured.

In order to simply illustrate FIG. 4″, the space correlation only,Corr_spat_mes_b(i:i, j:j+2δj, t:t) between the block b(i, j, t) and theblock b(i,j+2δj,t), and the space correlation Corr_spat_mes_b(i:i,j:j−δj, t:t) between the block b(i, j, t) and the block b(i−δi, j, t)have been represented.

The value of the block Bo(i, j, t) is then updated by selecting amongthe blocks b(i, j, t) and Bo(i, j, t), the one, the coordinates of whichhave the minimum value of these two sums.

Bo(i, j, t)<=>min[Corr_spat_mes_Bo_(—) b(i:i±x*δi, j:j±y*δj, t:t);Corr_spat_mes_(—) b(i:i±x*δi, j:j±y*δj, t:t)].

Similarly, in another embodiment illustrated in FIG. 4′″, the sum of thespace correlations Corr_spat_mes_Bo_b (i:i±x*δi,j: j±y*δj, t:t) betweena given block Bo(i, j, t) of the background image at a time (t) and atleast one block b(i±x*δi,j±y*δj,t) of the video frame image at the sametime t, preferably other than the block b(i, j, t), close to, andpreferably adjacent to said block b(i, j, t) is calculated.

As also illustrated in FIG. 4′″, the sum of the space correlationsCorr_spat_mes_Bo(i:i±x*δi, j:j±y*δj, t:t) between the block Bo(i, j, t)of the frame image at a time (t), and at least one block, close to, andpreferably adjacent to Bo(i±x*δi, j±y*δj, t) of the background image atthe same time (t) is also measured.

In order to simply illustrate FIG. 4′″, only the space correlationCorr_spat_mes_Bo(i:i, j:j+2δj, t:t) between the block Bo(i, j, t) andthe block Bo(i, j+2δj, t), and the space correlationCorr_spat_mes_Bo(i:i, j:j−δj, t:t) between the block Bo(i, j, t) and theblock Bo(i−δi, j, t) have been represented.

The value of the block Bo(i, j, t) is then updated by selecting amongthe blocks b(i, j, t) and Bo(i, j, t), the one, the coordinates of whichhave the minimum value of such two sums:

Bo(i, j, t)<=>min[Corr_spat_mes_Bo_(—) b(i:i±x*δi, j:j±y*δj, t:t);Corr_spat_mes_Bo(i:i±x*δi, j:j±y*δj, t:t)].

According to the invention, the measure of the spacecorrelation—illustrated in any one of FIGS. 4 to 4′″—can further becoupled to a measure of a time correlation, as regards the frames and/orthe background image, refer to FIG. 5.

In one embodiment, the sum of the space correlationsCorr_spat_mes_b_Bo(i:i±x*δi, j:j±y*δj, t:t) between a given block b(i,j, t) of one image at a time (t) and at least one block Bo(i±x*δi,j±y*δj, t) of the background image at a time t, preferably other thanthe block Bo(i, j, t), close to, and preferably adjacent to said blockBo(i, j, t), is calculated in the same way as the one illustrated inFIG. 4.

As also illustrated in FIG. 5, the sum of the space correlationsCorr_spat_mes_Bo(i:i±x*δi, j:j±y*δj, t−a:t) between the block Bo(i, j,t−a) of the background image at time (t−a) corresponding to said givenblock b(i, j, t−a) of the image at time (t−a), and at least a blockBo(i±x*δi, j±y*δj, t) of the background image at a time t, close to, andpreferably adjacent to the block Bo(i, j, t) is also calculated.

In order to simply illustrate the right part of FIG. 5, only the spacecorrelation Corr_spat_mes_Bo(i:i, j:j+δj, t−a:t) between the block Bo(i,j, t−a) and the block Bo(i, j+δj, t), the space correlationCorr_spat_mes_Bo(i:i−δi, j:j+δj, t−a:t) between the block Bo(i, j, t−a)and the block Bo(i−δi, j+δj, t), and the space correlationCorr_spat_mes_Bo(i:i−δi, j:j, t−a:t) between the block Bo(i, j, t−a) andthe block Bo(i−δi, j, t) have been represented.

The value of the block Bo(i, j, t) is then updated by selecting theblock corresponding to the minimum value of such sums:

Bo(i, j, t)<=>min[Corr_spat_mes_(—) b_Bo(i:i±x*δi, j:j±y*δj, t:t);Corr_spat_mes_Bo(i:i, j:j+δj, t−a:t);

Corr_spat_mes_Bo(i:i−δi, j:j+δj, t−a:t);

Corr_spat_mes_Bo(i:i−δi, j:j, t−a:t)].

As illustrated in FIG. 5′, the sum of the space correlationsCorr_spat_mes_Bo(i:i±x*δi, j:j±y*δj, t:t−a) between the block Bo(i, j,t) of the background image at a time (t) corresponding to said givenblock b(i, j, t) of the image at a time (t), and at least one blockBo(i±x*δi, j±y*δj, t−a) of the background image at time (t−a), close toand preferably adjacent to the block Bo(i, j, t) can be calculated.

In order to simply illustrate the right part of FIG. 5′, some spacecorrelations only, Corr_spat_mes_Bo(i:i±x*δi, j:j±y*δj, t:t−a) betweenthe block Bo(i, j, t−a) and the block Bo(i, j+δj, t), have beenrepresented.

Then the value of the block Bo(i, j, t) is updated by selecting theblock corresponding to the minimum value of these sums.

-   -   In other non illustrated embodiments, the sum of the space        correlations Corr_spat_mes_b(i:i±x*δi, j:j±y*δj, t:t−a) between        the block b(i, j, t) of the image of the video frame at a time        (t), and at least a block b(i±x*δi, j±y*δj, t−a) of the        background image at time (t−a), close to and, preferably        adjacent to, the block b(i, j, t) can be calculated. The sum of        the space correlations Corr_spat_mes_b(i:i±x*δi, j:j±y*δj,        t−a:t) between the block b(i, j, t−a) of the image of the video        frame at time (t−a), and at least a block b(i:i±x*δi,        j:j±y*δj, t) of the image of the video frame at a time (t),        close to and preferably adjacent to the block b(i, j, t) can        also be calculated.

Whatever the embodiment, two consecutive frames are preferably selected,so that a=δt.

Space correlations can be calculated by measuring the sum of theabsolute differences (SAD) between the block b(i, j) or Bo(i, j)considered and the close block(s) selected in the same frame, the sameframe at another time, on the same background image or the backgroundimage at another time.

For example, between a block b(i, j, t) and a set of blocks b(i±x*δi,j±y*δj, t) with {x, y}ε{1;2; . . . ;Δ}², the following can be set,according to specific directions (i.e. right, left, bottom, toprespectively):

droite_SADΔ[b(i, j, t)]=SAD[b(i, j, t), b(i−1, j, t)]+ . . . +SAD[b(i,j, t), b(i−Δ, j, t)]

gauche_SADΔ[b(i, j, t)]=SAD[b(i, j, t), b(i+1, j, t)]+ . . . +SAD[b(i,j, t), b(i+Δ, j, t)]

bas_SADΔ[b(i, j, t)]=SAD[b(i, j, t), b(i, j−1, t)]+ . . . +SAD[b(i, j,t), b(i, j−Δ, t)]

haut_SADΔ[b(i, j, t)]=SAD[b(i, j, t), b(i, j+1, t)]+ . . . +SAD[b(i, j,t), b(i, j+Δ, t)]

The minimum value of such sums SAD is kept as the measure of the spacecorrelation Corr_spat_mes_b(i:i±x*δi, j:j±y*δj, t:t) between a givenblock b(i, j, t) of an image at a time (t) and at least a preferablyadjacent block b(i±x*δi, j±y*δj, t) of the same frame.

The same principle is implemented to calculate the space correlationCorr_spat_mes_Bo(i:i±x*δi, j:j±y*δj, t:t) between the block Bo(i, j, t)of the background image at a time (t) and at least a block Bo(i±x*δi,j±y*δj, t) of the background image at the same time (t),

and, to calculate the space correlation Corr_spat_mes_Bo(i:i±x*δi,j:j±y*δj, t−a:t) between the block Bo(i, j, t−a) of the background imageat time (t−a) and at least a block Bo(i±x*δi, j±y*εj, t) of thebackground image at a time (t), close to, and preferably adjacent to theblock Bo(i, j, t).

or else, to calculate the space correlation Corr_spat_mes_b_Bo(i:i±x*δi,j:j±y*δj, t:t) between a given block b(i, j, t) of one image at a time(t) and at least a block Bo(i±x*δi, j±y*δj, t) of the background imageat a time t, preferably another one than the block Bo(i, j, t), closeto, and preferably adjacent to the block Bo(i, j, t).

The generic function of the space correlation calculation is thus namedCorr_spat_mes.

In one embodiment, the space correlation sum1 coupled to a timecorrelation measure is first measured on at least two frames.

For this purpose, the time correlation Corr_temp_mes_b(i, j, t) ismeasured—as described hereabove—between a given block b(i, j, t) of animage at a time (t) and the same block b(i, j, t−a) of an image atanother time (t−a), refer to FIG. 5.

For calculating sum1, the weight of the space correlation can also beweighted by a parameter λ, which may depend on time λ(t), so that

Sum1=Corr_temp_mes_(—) b(i, j, t)+λ(t)·Corr_spat_mes_(—) b_Bo(i:i±x*δi,j:j±y*δj, t:t)

For example λ is equal to δi, which is equal to δj, i.e. λ=2.

Similarly, the space correlation coupled to a time correlation measuresum2 is measured on at least two background images.

For this purpose, the time correlation Corr_temp_mes_Bo(i, j, t) ismeasured—as described hereabove—between a given block Bo(i, j, t) of thebackground image at a time (t) and the same block Bo(i, j, t−a) of thebackground image at another time (t−a).

The space correlation Corr_spat_mes_Bo(i:i±x*δi, j:j±y*δj, t−a:t) isalso measured between the block Bo(i, j, t−a) of the background image attime (t−a) corresponding to said given block b(i, j, t) of the image ata time (t), and at least an adjacent block Bo(i±δi, j±δj, t) of thebackground image at a time (t).

For calculating sum2, the weight of the space correlation can also beweighted by a parameter λ, which may depend on time λ(t), so that

Sum2=Corr_temp_mes_Bo(i, t)+λ(t)·Corr_spat_mes_Bo(i:i±x*δi, j:j±y*δj,t−a:t)

The value of the block Bo(i, j, t) is then updated by selecting theminimum value of these two values:

Bo(i, j, t)=min[sum1; sum2]

The block Bo(i, j, t) of the background image can thus be either theblock Bo(i, j, t−a) , or the block b(i, j, t).

Measuring the time or space correlation is carried out, for example, bymeasuring the difference in the values of the luminance of the pixel(s)of the blocks considered. Preferably, it is further executed bymeasuring the difference in the values of the chrominance of thepixel(s) of the blocks considered.

In one embodiment, the time or space correlation is determined bymeasuring the sum of the absolute differences (SAD) between the blockb(i, j, t) and the same block b(i, j, t±z*δt) on a plurality of frames,and by selecting the block, the absolute sum of which is the smallest ofall.

In order to simplify calculations, two consecutive frames t and t−1 arepreferably selected, so that a=δt (z=1).

According to the invention, the background image is updated on the fly.A block Bo(i, j) of the background image used for the space correlationcomputations may thus already have been updated during a previousiteration. Advantageously, each frame of blocks b(i, j) is scanned in afirst direction and in a second direction, i.e. a first series ofcorrelation computations is carried out starting from the first blockb(1, 1) at the top left of the frame to the last block b(N, M) at thebottom right of the frame; then a second series of correlationcomputations is carried out starting from the last block b(N, M) at thebottom right of the frame to the first block b(1, 1) at the top left ofthe frame.

Similarly, a dichotomized implementation can be provided for, so thateach frame of blocks b(i, j) is iteratively scanned with a gradient ofblock size, possibly each time in a first direction, then in a seconddirection.

For example, with an iteration I=2, the first scan is performed with asize of blocks b(i, j) equal to N1*M1; and the second scan is performedwith a size of blocks b(i, j) equal to N2*M2, so that N2<N1 and M2<M1.

Preferably, the second scan with the second size of the blocks isimplemented only on the blocks b(i, j) identified during the first scan,the space and/or time correlation of which is above a threshold.

In addition, as the background image is updated on the fly, the lattercan be displayed for the user, for example in a pre-view mode. In oneembodiment, the user can stop, whenever he/she wants, the implementationof the method as a function of the displayed background image.

According to the invention, a step of calculating the speed ofconvergence of the background image can also be provided for. Thecalculated speed is compared to a threshold and the scan of the blocksBo(i, j) of the background image is stopped if the speed is under thethreshold. The speed of convergence can be calculated for example bycounting the number of blocks Bo(i, j) updated relatively to the totalnumber of blocks b(i, j) of the frame image, equal to the total numberof blocks Bo(i, j) of the background image.

According to the invention, a step of checking the stationarity can alsobe provided for. For this purpose, if the stationarity of a block Bo(i,j), of the background image or a block Bo(i, j) of a frame is above athreshold, then an adjacent block Bo(i+δi, j+δj) is substituted for theblock Bo(i, j).

1-11. (canceled)
 12. A method of regenerating the background of adigital image of a video stream produced by a fixed camera and includinga plurality of time-sequential frames, with each one including arespective unit image, the method comprising: setting an initialbackground image in a memory; cutting the unit images of the videostream into blocks b(i, j, t) of at least one pixel; cutting thebackground image into corresponding blocks Bo(i, j, t) having the samesize and the same coordinates (i, j) as those of the unit images;selecting at least one block b(i, j, t), b(i, j, t−a), Bo(i, j, t−a),and Bo(i, j, t); and calculating the space correlation thereof, whereinthe space correlation comprises calculating the space correlation(Corr_spat_mes_b, Corr_spat_mes_b_Bo) between a given block b(i, j, t)of the frame image, at a time (t), and at least one block Bo(i±x*δi,j±y*δj, t) of the background image close to said block Bo(i, j, t), at atime (t), or at least one block Bo(i±x*δi, j±y*δj, t−a) of thebackground image close to said block Bo(i, j, t−a), at another time(t−a), or the space correlation (Corr_spat_mes_Bo, Corr_spat_mes_Bo_b)between a given block Bo(i, j, t) of the background image, at a time(t), and at least one block Bo(i±x*δi, j±y*δj, t) of the backgroundimage close to said block Bo(i, j, t), at a time (t), or at least oneblock Bo(i±x*δi, j±y*δj, t−a) of the background image close to saidblock Bo(i, j, t−a), at another time (t−a), or at least one blockb(i±x*δi, j±y*δj, t) of the frame image close to said block b(i, j, t),at a time (t), where (i±δi, j±δj) are the coordinates of a blockadjacent to a block having the coordinates (i, j) and x and y areintegers; and updating the background image according to the calculationof the space correlation; calculating the time correlation(Corr_temp_mes_b) between a given block b(i, j, t) of an image at a time(t) and the same block b(i, j, t−a) of an image or of several images ata preceding time (t−a); or calculating the time correlation(Corr_temp_mes_Bo) between a given block Bo(i, j, t) of an image at atime (t) and the same block Bo(i, j, t−a) of an image or of severalimages at a preceding time (t−a); and updating the background image alsoaccording to the time correlation; wherein the space correlation coupledto a measure of the time correlation is measured (sum1) on at least twoframes, where the weight of the space correlation is weighted by aparameter λ depending on time λ(t), so thatSum1=Corr_temp_mes_(—) b(i, j, t)+λ(t)*Corr_spat_mes_(—) b_Bo(i:i±x*δi,j:j±y*δj, t:t); wherein the space correlation coupled to a measure ofthe time correlation is measured (sum2) on at least two frames, wherethe weight of the space correlation is weighted by a parameter λdepending on time λ(t), so thatSum2=Corr_temp_mes_Bo(i, j, t)+λ(t)*Corr_spat_mes_Bo(i:i±x*δi, j:j±y*δj,t−a:t); and wherein the value of the block Bo(i, j, t) is updated byselecting the minimum value of both values (Sum1, Sum2); in which δrepresents a change, Corr_spat_mes_b_Bo (i:i±x*δi, j:j±δj, t:t) is thesum of the space correlations between block b(i,j,t) at a time (t) andat least one block Bo(i±x*δi, j±δ,t) of the background image at the sametime (t), Corr_spat_mes_Bo (i:i±x*δi, j:j±δj, t−a:t) is the sum of thespace correlation between block Bo(i, j, t−a) of the background image ata time (t−a) and at least a block Bo(i±x*δi, j±δj, t) of the backgroundimage at a time (t), with {x,y}ε{1;2; . . . ;Δ}².
 13. The method ofclaim 12, wherein the updating of the background image further comprisessubstituting the block b(i, j, t), b(i, j, t−a), Bo(i, j, t−a) or Bo(i,j, t) for the block Bo(i, j, t) by selecting among these the one havingthe minimum space correlation.
 14. The method of claim 12, wherein thespace correlation is calculated by measuring the sum of the absolutedifferences (SAD) between the blocks considered.
 15. The method of claim14, wherein the space correlation between a given block (b(i, j, t),Bo(i, j, t)) and a set of blocks (b(i±x*δi, j±y*δj, t), Bo(i±x*δi,j±y*δj, t)), with {x, y}ε{1;2; . . . . ;Δ}² is set along specificdirections, with the minimum value of the sums of the absolutedifferences (SAD) along each one of these specific directions beingretained as a measure of the space correlation.
 16. The method of claim12, wherein each frame of blocks b(i, j) or background image of blocks(Bo(i, j) is scanned at least in a first and in a second direction, sothat a first series of correlation computations is performed in thefirst direction; and a second series of correlation computations isperformed in the second direction.
 17. The method of claim 12, whereinthe size of the blocks b(i, j) or Bo(i, j) in the first direction isdifferent from that of the blocks b(i, j) or Bo(i, j) in the seconddirection.
 18. The method of claim 12, further comprising: computing thesum of measures of stationarity for the set of the blocks b(i, j, t) ofa frame at a time (t); comparing the sum of the measures of stationaritywith a first threshold (T1), and if the sum is under the first threshold(T1), then taking the frame into account for the space correlationcomputations.
 19. The method of claim 18, further comprising: comparingthe sum of the measures of stationarity with a second threshold (T2);and if the sum is above the second threshold (T2), then not taking theframe into account for the space correlation computations.
 20. Themethod of claim 12, further comprising: calculating the speed ofconvergence of the background image; comparing the calculated speed witha threshold; and stopping the scanning of the blocks Bo(i, j) of thebackground image if the speed is under the threshold.
 21. The method ofclaim 12, wherein the space correlation consists in further computingthe space correlation (Corr_spat_mes_b, Corr_spat_mes_b_Bo) between agiven block b(i, j, t) of the frame image at a time (t) and at least oneblock b(i±x*δi, j±y*δj, t) of the frame image close to said block b(i,j, t), at a time (t).
 22. A machine-readable media including programcode instructions operative to cause a processor to execute the steps ofthe method of claim 12.